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We study the discovery reach of the Tevatron and the LHC 
for detecting a Higgs boson (h), predicted in composite mod- 
els of the electroweak symmetry breaking or in supersymmet- 
ric theories, with an enhanced 6-quark Yukawa coupling via 
pp/pp — > bbh(—r bb) + X. Our analysis shows that studying 
this process at the Tevatron Run II or the LHC can provide 
strong constraints on these models. 

PACS number(s): 12.60.-i, 14.80. Cp, 14.65.Fy, 12.38.Bx 

The yet unverified Higgs sector in the standard model 
(SM) generates gauge boson masses via spontaneous elec- 
troweak symmetry breaking (EWSB) and fermion masses 
via Yukawa interactions. The large top quark mass, at 
the order of the EWSB scale, suggests that top may play 
a special role in the generation of mass. This occurs in 
models with dynamical top-condensate/topcolor scenar- 
ios |Q as well as in supersymmetric (SUSY) theories 
Since the bottom quark is the isospin partner of the top 
quark, its Yukawa coupling with a Higgs boson can be 
closely related to that of the top quark. As we will show, 
because of the small mass of bottom (m& ~ 4.5 GeV) 
relative to top (m t ~ 175 GeV), studying the b Yukawa 
couplings can effectively probe new physics beyond the 
SM. 

In this Letter, we study the detection of a Higgs boson 
(h) at hadron colliders predicted by models where bot- 
tom has an enhanced Yukawa coupling (yb)- We begin 
with a model-independent analysis for Higgs production 
associated with bb jets, pp/pp — > bbh — > bbbb + X, and 
determine the discovery reach of Tevatron Run II (a pp 
collider with v5 = 2 TeV) and LHC (a pp collider with 
y/S = 14 TeV) for using this production mode to probe 
models of dynamical EWSB and SUSY theories. 

1. Signal and Background 

To perform a model-independent analysis, we analyze 
the signal rate in terms of K, the factor of enhancement 
over the SM prediction for the hbb — > bbbb rate. By def- 
inition, K = |W(y b )sM| 2 BR(h -> bb)/BR{h -> bb) SM , 
where yb/(yb)su is the factor enhancing the h-b-b cou- 
pling relative to the SM coupling, and BR(/i — * bb) is the 
branching ratio. 

The relevant backgrounds to this signal at a hadron 
collider are QCD production of bbbb and bbjj, where 
j denotes a light quark or gluon, and production of 
Zbb — ► bbbb. We study the parton level hbb and Zbb rates 



at leading order (LO) with the Monte Carlo program PA- 
PAGENO H . The bbbb and bbjj processes are also stud- 
ied at the parton level, with matrix elements obtained 
from MADGRAPH @. In all cases, we include the full 
matrix elements for the gg as well as qq (and in the case of 
the bbjj background, qg and qg) initiated sub-processes, 
using the CTEQ4L parton distribution functions. We es- 
timate next-to-leading order (NLO) effects by including 
a fc-factor of 2 j| for the signal and all the background 
rates. We simulate the detector acceptance by requiring 
all four of the final state b (including b) quarks to have 
Pt > 20 GeV, rapidity |?7| < 2, and to be separated by a 
cone of AR > 0.4. 

In order to improve the signal to background ratio, we 
exploit the typical topology of the b quarks in the signal 
events. The h is radiated from one of the primary b 
quarks (by primary b quarks, we refer to those which do 
not result from the h decay), and thus there is typically 
one very energetic b quark with momentum on the order 
of the mass of h, rrih, balanced by the h and the other 
primary b, which is generally much softer. The h decays 
into a bb pair with typical transverse momentum on the 
order of m^/2. Thus, in order to enhance the signal, 
we order the transverse momenta of the b quarks, p^ > 
P { t > Pt' > Pt\ and require > 75 GeV, and pp 3) > 
40 GeV. These cuts moderately reduce the signal rate by 
about 23%, while providing a large 90% reduction of the 
QCD bbbb background. (Unless specified, our numerical 
results shown in this section are for detecting a scalar of 
mass 100 GeV at the Tevatron.) 

To further improve the signal to background ratio, 
we find the pair of b momenta with invariant mass 
that best reconstructs mn, and reject the event if the 
resulting invariant mass is more than Am/, from m/,, 
where Ato/j is the maximum of either our estimation 
of the experimental mass resolution (which is assumed 
to be 0.10 wih y/80Ge~V /rrih || or 10 GeV, whichever is 
higher) or the natural width (which is a calculable model- 
dependent quantity) of the scalar under study. We fur- 
ther require that the pr of the h (taken to be the pr of the 
sum of the four-momenta of the two b quarks which best 
reconstruct rrih) to be > 50 GeV, and that the b quark 
with largest px is not one of the two b quarks we have 
associated with the h decay. Because the signal generally 
produces an h whose px must balance p^!' , this does not 
affect the signal rate, but further reduces the QCD bbbb 
background by about 90%. 
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Finally, because the signal events typically produce a 
primary b and b on opposite sides of the detector, we 
require that the two b's which were not associated with 
the h decay have invariant mass Mbb > 50GeV. This 
further reduces the QCD bbbb background by about 60%. 
It is interesting to note that despite the fact that the 
gg — > hbb subprocess composes more than 95% of the 
signal rate at the Tevatron if one applies only the basic 
acceptance cuts, the situation is entirely different for our 
optimized search strategy, for which the qq — > hbb process 
makes up 93% of the signal. 

In order to reduce the huge bbjj background, we re- 
quire all four b quarks to be tagged, and estimate a 60% 
probability for tagging a b quark satisfying the cuts de- 
scribed above. We allow a 0.5% probability for a light 
quark or gluon to be mistaken for a b quark. After apply- 
ing this requirement, we find that the bbjj background 
is very small. One can slightly improve the bounds ob- 
tained by requiring only three of the four jets to be b- 
tagged, thus saving more of the signal rate. 

In Tab. 1 we present the resulting number of events 
at the Tevatron (assuming 2 fb -1 of integrated lumi- 
nosity) for the hbb signal (assuming rrih = 100 GeV, 
a mass resolution of 10 GeV, and an enhancement of 
K = (mt/rrib) 2 — (39) 2 ) as well as the background pro- 
cesses. In the second column we present the number of 
events after imposing the minimal acceptance cuts de- 
scribed above. In the third column, we show the number 
of events remaining after employing our optimized search 
strategy. In columns four and five we present the results 
after requiring 3 or 4 6-tags, respectively. As can be seen 
from the table, our optimized result combined with 3 (4) 
6-tags shows a significance (calculated as the number of 
signal events divided by the square root of the number 
of background events) of 43.0 (38.3). Thus it is possible 
to use Tevatron data to exclude this choice of parame- 
ters at better than 99%. We obtain similar results for 
m h = 100 (200, 300, 400) GeV, finding a total number of 
background events (assuming 4 6-tags and a Higgs with 
width less than the experimental mass resolution) of 59 
(4, 2, 0) for a 10 fb -1 Tevatron and 2.5 xlO 5 (4.7 xlO 4 , 
1.7 xlO 4 , 900) for a 100 fb" 1 LHC. 



Process 


Acceptance 


Optimized 


3 6-tagged 


4 6-tagged 


hbb 


1690 


979 


465 


127 


Zbb 


300 


54 


26 


7 


bbbb 


5071 


33 


16 


4 


bbjj 


5 x 10 7 


2 x 10 4 


73 






Table 1. The signal and background events for 2 fb of Tevatron 
data, assuming m/, = 100 GeV, Amj, = 10 GeV, and K = 39 2 . 



From the results of our study we can determine the 
minimal value of K, AT m i n , required to give a 95% C.L. de- 
viation from the background for a given to^, using Gaus- 
sian statistics when the number of background events is 



more than 10, and Poisson statistics if it is less than 10. 
From this we determine the curve of K mm versus mu for 
the Tevatron Run II with 2, 10, and 30 fb -1 , and for 
the LHC with 100 fb -1 . Being conservative, we require 
4 6-tags in Sees. 2 and 3 to interpret these results in the 
context of the models with dynamical EWSB and the 
Minimal Supersymmetric SM (MSSM). 

2. Implications for Models of Dynamical EWSB 

Examples of the strongly interacting EWSB sector 
with composite Higgs boson(s) are top-condensate and 
topcolor models Q| , in which new strong dynamics asso- 
ciated with the top quark play a crucial role for top and 
W, Z mass generation. A generic feature of these models 
is the naturally large Yukawa coupling of bottom (yb), 
of the same order as that of top (m ~ 1), due to the 
infrared quasi-fixed-point structure J 7 ] and their particu- 
lar boundary conditions for (?/&, y t ) at the compositcness 
scale. 

Two-Higgs-Doublet Extension of the BHL Model 

The effective theory of the top-condensate model is the 
SM without its elementary Higgs boson, but instead, 
with 4-Fermi interaction terms induced from (unspeci- 
fied) strong dynamics at a high scale A. The minimal 
BHL top-condensate model with three families (|], con- 
tains only one type of 4-Fermi vertex for <it> con- 
densation which generates the masses for top, W, and 
Z, but the predicted m t is too large to reconcile with 
experiment. Thus, we consider the two Higgs doublet 
extension (2HD E) fl9| as an example (which, with some 
improvements p],|lO|, is expected to produce an accept- 
able mt), and examine its prediction for the hbb rate. 
The 4-Fermi interactions of the 2HDE model produce 
condensates in both tt and bb channels, which generate 
the EWSB and induce two composite Higgs doublets <E>t 
and so that the Yukawa interactions take the form of 
y t {^L^ttR + h.c.) and y b (^ L <S>bb R + hue). Here, ^ L is 
the left-handed quark doublet and tp is the right-handed 
top, etc. This model predicts yt(A) = y&(A) > 1 at 
the scale [i = A [0J|]. Therefore, yt(^) ~ yb(n) for 
any ji < A, because the renormalization group equa- 
tions governing the running of yt and yb are similar ex- 
cept for the small difference in the the t and b hyper- 
charges 0,0]. Due to the dynamical <it> and <bb> 
condensation, the two composite Higgs doublets develop 
VEVs «E> t >= (v t ,0) T /V2 and <$ b >= {0,v b f /y/2. Since 
™h(= ybVb/V2) is fixed by the experimental value at 
H = TOb and yb is about equal to j/t(~l), this model 
predicts a large tan/3 = v t /v b ^ m t /m b = 0(39) 3> 1 . 

The 2HDE has three neutral scalars, the lightest 
(with enhanced b coupling) being the pseudoscalar P(= 
V2[sin/3Im$° + cos/3Im$t]), whose mass (Mp) is less than 
about 233 GeV for A = 10 15 GeV @. Given j/ b and_M P , 
one can calculate the production rate of Pbb(-^ bbbb) at 
hadron colliders, and thus for a given Mp one can de- 
termine the minimal yb value needed for the Tevatron 
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and LHC to observe the signal. As shown in Fig. la, the 
Tevatron data with 2fb _1 will exclude such a model at 
95%C.L. 

In Topcolor Assisted Technicolor 

The topcolor-assisted technicolor models (TCATC) Jll]] 
postulate the gauge structure Q = SU{2>)\ ® SU(3) 2 ® 
U (l)i ® U(l)-2 ® SU{2)l at the scale above A to explain 
the dynamic origin of the 4-Fermi coupling(s) in top- 
condensate models. At A~0(1) TeV, Q spontaneously 
breaks down to SU(3) C ® U(1) Y <S> SU(2) L , and ad- 
ditional massive gauge bosons are produced in color 
octet (B a ) and singlet [Z') states. Below the scale 
A = mm(MB,Mz>), the effective 4-Fermi interactions 
are generated in the form of 

Uf = -&[(« + fst) *£tfl*K*i + (« - wfc) • 
In the low energy effective theory at the EWSB scale, 
two composite Higgs doublets are induced with the 
Yukawa couplings yt — W4<k(k + 2ni/9N c ) and yb — 
\J — Ki /9N C ) , where n and k\ originate from the 
strong SU(3)i and t/(l)i dynamics, respectively. It is 
clear that, unless K\ is unnaturally larger than k, yb is 
expected to be only slightly below y t . The £7(1) 1 force is 
attractive in the <tt> channel but repulsive in the <bb> 
channel, thus t, but not b, acquires a dynamical mass, 
provided y b (A) < y clit = y/8ir 2 /3 < y t (A) . (In 
this model, b acquires a mass mainly from the topcolor 
instanton effect PI-) Furthermore, the composite Higgs 
doublet <i>t, but not develops a VEV, i.e., vt ^ and 
v b = 0. 



1 1 1 1 ' 

- (0) 

Theor. Upper Limit 
(A= 1 1s GeV) 




1 1 

A= 1 TeV 


(b> ; 




3TeV 


5ToV 






; A= 1 OTeV 




tanl3 = 39 








Tevatron 




Tevatron 




2fb"' 
: /I \Ofb-' 






/30fb _1 ~ 


/30fb"' 

LHC 

lOOfb -1 




LHC 


1 OOfb - .'. 



' M P (GeV) 



M BB (GeV) 



FIG. 1. Discovery reach of Tevatron and LHC for the mod- 
els of 2HDE (a) and TCATC (b). Regions above the curves 
can be discovered at 95%C.L. In (b), the straight lines in- 
dicate yt{n = rn t ) values for typical values of the topcolor 
breaking scale A, and yb is predicted to be very close to yt- 

In TCATC, the topcolor interaction generates m t , but 
is not responsible for the entire EWSB. Thus, A can be as 
low as 0(1 — 10) TeV (which avoids the severe fine-tuning 
needed in the minimal models [|||] ) , and correspondingly, 



v t = 64 - 88 GeV for A = 1 - 5 TeV by the Pagels- 
Stokar formula. The smaller Vt value (as compared to 
v = 246 GeV) predicted in the TCATC model makes the 
top coupling to $t stronger, i.e., yt = 2.8 — 3.9 at 
fi = mt, than in the SM (y t ~ 1). As explained above, 
this requires y b to be also large. Thus, the neutral scalars 
hb and A b in the doublet $ , which are about degenerate 
in mass, have an enhanced coupling to the 6-quark. 

In Fig. lb, we show the minimal value of y b /(y b )sM 
needed to observe the TCATC model signal as a function 
of Mh b ■ As shown, if Mh b is less than about 400 GeV, the 
Tevatron Run II data can effectively probe the scale of 
the topcolor breaking dynamics, assuming the TCATC 
model signal is observed. If the signal is not found, the 
LHC can further explore this model up to large Mh b ■ For 
example, for Mh b = 800 GeV, the required minimal value 
oiyb/{y b )sM is about 4.9 at 95%C.L. Similar conclusions 
can be drawn for a recent left-right symmetric extension 
[UI of the top-condensate scenario, which also predicts a 
large 6-quark Yukawa coupling. 

3. Implications for Supersymmetric Models 

The EWSB sector of the MSSM model includes two 
Higgs doublets with a mass spectrum including two 
neutral CP-even scalars h° and H°, one CP-odd pseu- 
doscalar A and a charged pair H ± . The Higgs sector is 
completely determined at tree level by fixing two param- 
eters, conventionally chosen to be tan (3 and the pseu- 
doscalar mass rriA fljj • At loop level, the large radiative 
corrections to the Higgs boson mass spectrum are domi- 
nated by the contributions of top and stop (top-squark) 
in loop s []14{ |. In this study we employ the full one loop 
results |15| to generate the Higgs mass spectrum assum- 
ing all sfermion masses, /z, scalar tri-linear parameters, 
and SU(2) l gaugino masses at the electro- weak scale are 
equal to 500 GeV. We find that our results are fairly 
insensitive to this choice of parameters. 

tan f3 is a free parameter of the MSSM. The current low 
energy bound gives mh,mA > 75 GeV for tan/3 > 1 [^6| . 
Since the couplings of h°-b-b, H°-b-b and A°-b-b are pro- 
portional to sina/cos/3, cos a/ cos (3 and tan/3, respec- 
tively, they can receive large enhancing factor when tan (3 
is large. This can lead to detectable bbbb signal events at 
the Tevatron, as previously studied in Ref. ]l7[] . Here, 
we improve the calculations in the signal and the back- 
ground rates (c.f. Sec.l) and the prediction of the MSSM 
by including large loop corrections. We calculate the en- 
hancement factor K predicted by the MSSM for given 
values of tan/3 and m^. In Fig. 2 we present the discov- 
ery reach of the Tevatron and the LHC, assuming a stop 
mass of 500 GeV and that all the superparticles are so 
heavy that Higgs bosons will not decay into them at tree 
level. The BR for h — > bb is close to one for most of the 
parameter space above the discovery curves. Moreover, 
for tan (3 ^> 1, the h° is nearly mass-degenerate with the 
A (if vriA is less than ~120 GeV) and otherwise with H°. 



3 



We thus include both scalars in the signal rate provided 
their masses differ by less than Arrih- The MSSM can 
also produce additional bbbb events through production 
of h°Z — > bbbb and h°A° — > bbbb, however these rates are 
expected to be relatively small when the Higgs-bottom 
coupling is enhanced, and the resulting kinematics are 
different from the hbb signal. Thus we conservatively do 
not include these processes in our signal rate. 

From Fig. 2 we deduce that if a signal is not found, 
the MSSM with tan/3 > 23 (16, 12) can be excluded 
for rriA up to 200GeV at the 95% C.L. by Tevatron data 
with a luminosity of 2 (10, 30) fb _1 ; while the LHC can 
exclude a much larger ttia (for ttia = 800 GeV, the mini- 
mal value of tan (3 is about 7.4). These Tevatron bounds 
thus improve a recent result obtained by studying the 
bbrr channel [[L8|| . We note that studying the hbb mode 
can probe an important region of the tan (3-m,A plane 
which is not easily covered by other production modes 
at hadron colliders, such as pp —* tt + h(— > 77) + X 
and pp — > h(— * ZZ*) |X @. Also, in this region of 
parameter space the SUSY Higgs boson h° is clearly dis- 
tinguishable from a SM one. 
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FIG. 2. The regions above the curves in the tan/3-mA 
plane can be probed at Tevatron and LHC with a 95% C.L.. 

The above results provide a general test for many 
SUSY models, for which the MSSM is the low energy ef- 
fective theory. In the MSSM, the effect of SUSY breaking 
is parametrized by a large set of soft-breaking (SB) terms 
(^0(100)), which in principle should be derived from 
an underlying model. We discuss, as examples, the Su- 
pergravity and Gauge-mediated (GM) models with large 
tan (3. In the supergravity-inspired model |2(| the SUSY 
breaking occurs in a hidden sector at a very large scale, 
of O(10 10 - n )GeV, and is communicated to the MSSM 
through gravitational interactions. In the simplest model 
of this kind, all the SB parameters are expressed in terms 
of 5 universal inputs. The case of large tan/3, of O(10), 
has been examined within this context [pl| , and it was 
found that in such a case m^^lOO GeV. Hence, these 
models can be cleanly confirmed or excluded by measur- 
ing the bbbb mode at the Tevatron and LHC. 

The GM models assume that the SUSY-breaking scale 
is much lower, of O(10 4-5 ) GeV, and the SUSY breaking 



is communicated to the MSSM superpartners by ordi- 
nary gauge interaction p2{ . This scenario can predict 
large tan/3 (~ 30-40). However, in some models, it fa- 
vors mA ~ 400 GeV [||) , which would be difficult to test 
at the Tevatron, though quite easy at the LHC. Never- 
theless, in some other models, a lighter pseudoscalar is 
possible (for instance, tan/3 = 45 and ttia — 100) 
and the bbbb mode at hadron colliders can easily explore 
such a SUSY model. 

In conclusion, the large QCD production rate at a 
hadron collider warrants the detection of a light scalar 
with large h-b-b coupling. At LEP-II and future e + e~ lin- 
ear colliders, because of the large phase space suppression 
factor for producing a direct 3-body final state as com- 
pared to first producing a 2-body resonant state, the bbA 
and bbh rates predicted by the MSSM are dominated by 
the production of Ah and hZ pairs via electroweak inter- 
action. Hence, the e + e~ collider is less able to directly 
probe the h-b-b coupling. 
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